Estimating the information available from colored surfaces in natural scenes

The information preserved in identifying surfaces solely by their color can be quantified by measures defined by Shannon, including capacity and mutual information. The aim of this study was (1) to determine whether the capacity of an additive Gaussian channel provides an asymptotic upper bound to mutual information estimated for natural scenes under different illuminants, and (2) to explore the effect of different color representations on mutual-information estimates. Introduction Color is an imperfect code for representing surfaces in a scene. This is because the number of degrees of freedom in a sensor system, three for the cones in the normal human eye or a typical camera, is smaller than the number of degrees of freedom needed to specify different spectra [1]–[4]. Surfaces that match under one light therefore need not match under another: the phenomenon of metamerism. As a consequence, identifying surfaces on the basis of their color will lead to errors when the illuminant on scene changes. Such errors represent a loss of information. How much information, then, is preserved when surfaces are coded solely by their color? One way to address this question is to estimate a quantity such as Shannon’s mutual information [5, 6]. But making a direct estimate based on the probabilities involved leads to difficulties when those probabilities are small [7]. Instead, an asymptotic upper bound to the mutual information may be obtained from an analysis of the capacity of an additive Gaussian channel [8, 9]. The objective of the present work was to test whether capacity is an asymptotic upper bound to mutual information when estimated with samples of increasing size from natural scenes, and to explore how different color spaces [10, 11], color-difference formulae [12], and spectral sharpening [13] affect mutual-information estimates. Methods Hyperspectral Images Scene reflectances were drawn from a set of eight hyperspectral images [8] (three of which are shown in Fig. 1). These images were from rural and urban areas in the Minho region of Portugal. The size of the images was approximately 1344× 1024 pixels, and spectra at each pixel were defined at 10-nm wavelength intervals over 400–720 nm. Further technical details are available elsewhere [14, 8, 9]. Representation of Scenes In computational simulations, scenes were illuminated successively by daylights of correlated color temperatures of 4000 K, 6500 K and 25000 K. For each of the three illuminants, the spectrum of the reflected light at each pixel was converted to tristimulus values and then to corresponding values (Xc,Yc,Zc), calculated with the CMCCAT2000 chromatic-adaptation transform [10] under the assumption of full adaptation and with D65 as reference. A version of CMCCAT2000 with sharp chromaticadaptation transform [13, 15] was also used. The tristimulus values (Xc,Yc,Zc) were transformed to CIELAB values (L∗,a∗,b∗) with D65 as reference. Since CIELAB space is well known to be perceptually non-uniform [11], the color-difference formulae CIEDE2000 and CMCDE [12, 17] were each used to evaluate the differences in color-code values, in addition to the Euclidean distance. Information-theoretic Measures and Estimates For a particular scene, index the pixels in image 1 of the scene under illuminant e1 by variable X and in image 2 of the scene under illuminant e2 by variable Y . Suppose that in some task the probability of a particular pixel x in image 1 being chosen is p(x) = P{X = x} and of a particular pixel y in image 2 being chosen is p(y) = P{Y = y}, where there are N pixels in each image. The degree of uncertainty associated with each image can be quantified by the entropy [5, 6] defined as follows: H (X) = − N ! x=1 p(x) log p(x), (1) with a similar expression for H(Y ). If the conditional probability p(x|y) = P{X = x|Y = y} is known, then the conditional entropy is given by the expression: H (X |Y ) = − N ! y=1 p(y) N ! x=1 p(x|y) log p(x|y). (2) This quantity represents the uncertainty about image 1 given image 2. The mutual information can then be expressed as the difference of the two entropies: I (X ;Y ) =H (X)−H (X |Y) . (3) Mutual information represents the reduction in uncertainty about image 1 given image 2. If the basis of the logarithm is 2, then the mutual information is expressed in bits (the convention adopted here). As noted earlier, estimating I(X ;Y ) directly from the probabilities leads to difficulties [7]. In principle, each pixel y under illuminant e2 may be coded with value (L2,a ∗ 2,b ∗ 2), say, and compared with each pixel x under illuminant e1 coded with value (L1,a ∗ 1,b ∗ 1) and then matched according the closest code value. Other color codes based on tristimulus values (Xc,Yc,Zc) or transformed CIELAB values (L′,C′,H ′) [12], with either CIEDE2000 or CMCDE color-difference formulae, can be used. An alternative approach is to consider the distributions of the color-code values (L∗,a∗,b∗) under the two illuminants. An asymptotic upper boundC on the mutual information I(X ;Y ), for large enough N, can be estimated from an analysis of the capacity of an additive Gaussian channel [6] by considering the differences in code values under the two illuminants as noise. The Figure 1. Examples of pictures obtained with a hyperspectral imaging system [8]. capacity of this Gaussian channel has a formulation in terms of the covariance matrix of the code values of the scene under e1 and covariance matrix of the noise [9]. Thus, let "1 be the covariance matrix of the code values (L1,a ∗ 1,b ∗ 1), and "# the covariance matrix of the code value differences (#L∗,#a∗,#b∗). Then the capacity of the channel is given by the following: C = 1 2 log ( |"1+"#| |"#| )